APEC8211: Recitation 2

class: center, middle, inverse, title-slide

.title[
# APEC8211: Recitation 2
]
.author[
### Shunkei Kakimoto
]

---

class: middle

.remark-slide-content.hljs-github h1 {
  margin-top: 5px;  
  margin-bottom: 25px;  
}

.remark-slide-content.hljs-github {
  padding-top: 10px;  
  padding-left: 30px;  
  padding-right: 30px;  
}

.panel-tabs {
  
  color: #841F27;
  margin-top: 0px;  
  margin-bottom: 0px;  
  margin-left: 0px;  
  padding-bottom: 0px;  
}

.panel-tab {
  margin-top: 0px;  
  margin-bottom: 0px;  
  margin-left: 3px;  
  margin-right: 3px;  
  padding-top: 0px;  
  padding-bottom: 0px;  
}

.panelset .panel-tabs .panel-tab {
  min-height: 40px;
}

.remark-slide th {
  border-bottom: 1px solid #ddd;
}

.remark-slide thead {
  border-bottom: 0px;
}

.gt_footnote {
  padding: 2px;  
}

.remark-slide table {
  border-collapse: collapse;
}

.remark-slide tbody {
  border-bottom: 2px solid #666;
}

.important {
  background-color: lightpink;
  border: 2px solid blue;
  font-weight: bold;
}

.remark-code {
  display: block;
  overflow-x: auto;
  padding: .5em;
  background: #ffe7e7;
}

.remark-code, .remark-inline-code { 
  font-family: 'Source Code Pro', 'Lucida Console', Monaco, monospace;font-size: 50%;
}

.hljs-github .hljs {
  background: #f2f2fd;
}

.remark-inline-code {
  padding-top: 0px;
  padding-bottom: 0px;
  background-color: #e6e6e6;
}

.r.hljs.remark-code.remark-inline-code{
  font-size: 0.9em
}

.left-full {
  width: 80%;
  float: left;
}

.left-code {
  width: 38%;
  height: 92%;
  float: left;
}

.right-plot {
  width: 60%;
  float: right;
  padding-left: 1%;
}

.left6 {
  width: 60%;
  height: 92%;
  float: left;
}

.left5 {
  width: 49%;
  
  float: left;
}

.right5 {
  width: 49%;
  float: right;
  padding-left: 1%;
}

.right4 {
  width: 39%;
  float: right;
  padding-left: 1%;
}

.left3 {
  width: 29%;
  height: 92%;
  float: left;
}

.right7 {
  width: 69%;
  float: right;
  padding-left: 1%;
}

.left4 {
  width: 38%;
  float: left;
}

.right6 {
  width: 60%;
  float: right;
  padding-left: 1%;
}

ul li{
  margin: 7px;
}

ul, li{
  margin-left: 15px; 
  padding-left: 0px; 
}

ol li{
  margin: 7px;
}

ol, li{
  margin-left: 15px; 
  padding-left: 0px; 
}

</style>

.full-width {
    display: flex;
    width: 100%;
    flex: 1 1 auto;
}
</style>

.blockquote p {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h1:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h2:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h3:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}
.blockquote > h4:first-of-type {
  margin-top: 0px;
  margin-bottom: 5px;
}

.text-shadow {
  text-shadow: 0 0 4px #424242;
}
</style>

.scroll-box-8 {
  height:8em;
  overflow-y: scroll;
}
.scroll-box-10 {
  height:10em;
  overflow-y: scroll;
}
.scroll-box-12 {
  height:12em;
  overflow-y: scroll;
}
.scroll-box-14 {
  height:14em;
  overflow-y: scroll;
}
.scroll-box-16 {
  height:16em;
  overflow-y: scroll;
}
.scroll-box-18 {
  height:18em;
  overflow-y: scroll;
}
.scroll-box-20 {
  height:20em;
  overflow-y: scroll;
}
.scroll-box-24 {
  height:24em;
  overflow-y: scroll;
}
.scroll-box-30 {
  height:30em;
  overflow-y: scroll;
}
.scroll-output {
  height: 90%;
  overflow-y: scroll;
}

</style>

# Outline

Review some concepts related to random variables

[1. CDF, PDF, PMF](#dist)

[2. Mean, variance and covariance](#mean)

[3. Jensen's inequality](#Jensen)

---
name: dist
# CDF, PDF, PMF, Quantile function
.content-box-red[**CDF**]
+ Definition: <span style="color:red">The CDF of a random variable X is `$F(x) = Pr[X \leq x]$`</span>
+ **Verbally**: CDF `$F(x)$` tells us the probability of the event that random variable `$X$` is less than a value `$x$`.

.left5[
<img src="data:image/png;base64,#recitation2_slides_files/figure-html/unnamed-chunk-5-1.png" width="100%" style="display: block; margin: auto;" />
]
.right5[
<img src="data:image/png;base64,#recitation2_slides_files/figure-html/unnamed-chunk-6-1.png" width="100%" style="display: block; margin: auto;" />
]

---
.content-box-red[**Probability mass function (Discrete random variables)**]
+ Definition: `$\color{red}{\pi(x) = Pr[X = x]}$`
+ **Verbally**: The probability that `$X$` equals the value `$x$`

<br>

.content-box-red[**Probability density function (Continuous random variables)**] 
+ Definition:  `$\color{red}{f(x) = \frac{d}{dx}F(x)} \quad ( = \displaystyle \lim_{x\to\infty} \frac{F(x+h)-F(x)}{h})$`
+ **Verbally**: Density function is a

<br>

.content-box-red[**Theorem 2.3: Properties of a PDF**]

A function f(x) is a density function **if and only if**

`$$\begin{cases}
f(x) \ge 0 \text{ for all } x \\
\int_{x=-\infty}^\infty x\,dx = 1
\end{cases}$$`

---
class: middle

.content-box-green[**Relationship between CDF and PDF**]

# Prepare results

---
# Mean, Variance and Covariance:
.content-box-red[**Mean: E[X]**]
+ Definition: <span style='color:red'>The mean of `$X$` is `$E[X]$`</span>
+ How to calculate it?:
  * for discrete `$X$`?
  * for continuous `$X$`?

+ .content-box-green[Visualization]

---
# Mean, Variance and Covariance:
.content-box-red[**Variance: Var[X]**]
+ Definition: <span style='color:red'>The variance of `$X$` is `$Var[X]=E[(X-E[X])^2]$`</span>
+ How to calculate it?
  * for discrete `$X$`?
  * for continuous `$X$`?

+ .content-box-green[Visualization]

---
# Mean, Variance and Covariance:
.content-box-red[**Coariance: Cov[X, Y]**]
+ Definition: <span style='color:red'>The covariance between X and Y are</span>

$$
\color{red}{Cov(X, Y) = E[(X-E[X])((Y-E[Y]))]}
$$
+ How to calculate it?
  * for discrete `$X$`?
  * for continuous `$X$`?

+ .content-box-green[Visualization]

---
class: middle

# Expectation, Variance and Covariance as operators
In Econometrics analysis, we often use `$E[\,]$`, `$Var[\,]$`, `$Cov[\,]$` as operators (i.e., functions). The following are useful knowledge that you should know.

---
class: middle

.content-box-red[**Expectation**]

<span style="color:blue">Expectation is a linear operator.</span>

That is, for any constants `$a$` and `$b$` and any random variables `$X$` and `$Y$`:

(1) `$E[a+bX]=a+bE[X]$`

(2) `$E[X+Y]=E[X]+E[Y]$`

+ Note: Linearity of expectation only applies to the sum of random variables (random variables are in the linear function)

.content-box-green[**Question**]
+ True or False: `$E[X+X^2]=E[X]+E[X^2]$`?
+ True or False: `$E[XY]=E[X]E[Y]$`

---
class: middle

.content-box-red[**Variance**]

<span style="color:blue">Variance is not a linear operator except for special case.</span>

For any constants `$a$` and `$b$` and any random variables `$X$` and `$Y$`:

(1) `$Var[X]=E[X^2] - (E[X])^2 \quad (\text{Simply, another definition of } Var[X]$` )

(2) `$Var[aX]=a^2Var[X]$`

(3) `$Var[a+bX]=b^2Var[X]$`

(4) `$Var[X+Y]=Var[X]+Var[Y] + Cov(X, Y)$`

.content-box-green[**Question**]
+ In what condition does `$Var[X+Y]=Var[X]+Var[Y]$` hold?

.content-box-green[**Exercise**]
+ Let's prove (1) and (2)

---
class: middle

.content-box-red[**Visualization**]

For any constants `$a$` and `$b$` and any random variables `$X$` and `$Y$`:

(1) `$Cov[X, Y]=E[XY]-E[X]E[Y] \quad (\text{Simply, another definition of } Cov[X, Y]$`)

(2) `$Cov[aX, Y]=aCov[X,Y]$`

(3) `$Cov[X, a+bY]=bCov[X,Y]$`

.content-box-green[**Question**]
+ In what condition does `$Cov[X, Y]=0$` hold?

.content-box-green[**Exercise**]
+ Let's prove (1)-(3)

---

# Jensen's inequality
In the previous slide, we saw `$Var[X]=E[(X-E[X])^2]=E[X^2] - (E[X])^2$`. So, `$Var[X] \ge 0$` (by the way, `$Var[X]=0$` if and only if `$X$` is degenerate).

So, 
`$$E[X^2] - (E[X])^2 \ge 0$$`
<p style="text-align: center;">or</p>
`$$(E[X])^2 \leq E[X^2]$$`
Define `$g(x)=x^2$`. The it is written as
`$$g(E[X]) \leq E[g(X)]$$`

Generally, 
`$$\begin{align*}
g(E[X]) \leq E[g(X)] \quad &\text{if } g(x) \text{ is a convex function} \\
E[g(X)] \leq g(E[X]) \quad &\text{if } g(x) \text{ is a concave function}
\end{align*}$$`

---
.content-box-green[**Visualization**]

.panelset[

.panel[.panel-name[Example 1 : g(x) is convex]
.left5[

```r
set.seed(356)
x <- runif(1000, 0, 10)

# /*===== Convex case: g(X)=X^2 =====*/
y <- x^2

figure_ex1 <-
  ggplot()+
  geom_point(aes(x = x, y = y))+
  # --- E[X] --- #
  geom_vline(xintercept = mean(x), color = "red", linetype = "dashed")+
  annotate("text", x = mean(x)+1, y = 0.01,
    label = paste0("E[X]=", round(mean(x), 1)), size = 3, color = "red") +
  # --- E[g(X)] --- #
  geom_hline(yintercept = mean(y), color="blue", linetype = "dashed")+
  annotate("text", x = 1, y = mean(y)+5,
    label = paste0("E[g(X)]=", round(mean(y), 1)), size = 3, color = "blue") +
  # --- g(E[X]) --- #
  geom_hline(yintercept = mean(x)^2, color="green", linetype = "dashed")+
  annotate("text", x = 1, y = mean(x)^2-5,
    label = paste0("g(E(X))=", round(mean(x)^2, 1)), size = 3, color = "green") +
  theme_bw()
```
  ]
.right5[
<img src="data:image/png;base64,#recitation2_slides_files/figure-html/unnamed-chunk-12-1.png" width="100%" style="display: block; margin: auto;" />

`$$\color{lightgreen}{g(E[X])} \leq \color{blue}{E[g(X)]}$$`
  ]
]

.panel[.panel-name[Example 2: g(x) is concave]
.left5[

```r
# /*===== Convex case: g(X)=X^(1/2) =====*/
y <- x^(1/2)

figure_ex2 <-
  ggplot()+
  geom_point(aes(x = x, y = y))+
  # --- E[X] --- #
  geom_vline(xintercept = mean(x), color = "red", linetype = "dashed")+
  annotate("text", x = mean(x)+0.8, y = 0.01,
    label = paste0("E[X]=", round(mean(x), 1)), size = 3, color = "red") +
  # --- E[g(X)] --- #
  geom_hline(yintercept = mean(y), color = "blue", linetype = "dashed")+
  annotate("text", x = 1, y = mean(y)-0.2,
    label = paste0("E[g(X)]=", round(mean(y), 2)), size = 3, color = "blue") +
  # --- g(E[X]) --- #
  geom_hline(yintercept = mean(x)^(1/2), color = "green", linetype = "dashed")+
  annotate("text", x = 1, y = mean(x)^(1/2)+0.2,
    label = paste0("g(E(X))=", round(mean(x)^(1/2), 2)), size = 3, color = "green") +
  theme_bw()
```
  ]
.right5[
<img src="data:image/png;base64,#recitation2_slides_files/figure-html/unnamed-chunk-14-1.png" width="100%" style="display: block; margin: auto;" />
`$$\color{blue}{E[g(X)]} \leq \color{lightgreen}{g(E[X])}$$`
  ]
]
]